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ABSTRACT relationship of the foreign language teacher and 

the researcher is explored in this report f 

on three main aspects; (1) locating research, (2) evaluating 
research, and (3) conducting research. An overview 
teacher as potential researcher introduces the 

Distinctions between overview reading and topical reading includ 
°et«-ncer?o specific journals, periodicals, and other sources 
considered to be important to language teachers. 

identify the most important criteria in evaluating typical research 
undertakngs concentrate on demonstrating how 

the applicability and validity of research for their particular 
needs^ Advice for the teacher conducting research stresses the need 
to limit the scope of the problem and the careful development of an 
operational plan. A bibliography is included. (RL) 
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research and development in its own area of education. The ERIC 
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I. INTRODUCTION 



Why should the foreign-language teacher become interested and 
involved in language- teaching research? Certainly, language teach- 
eis can “do without” research in the sense that even a rudimentary 
background in this area is virtually never a prerequisite to em- 
ploy nient or advancement. Certainly also, the already gi’eat demand 
made on the teacher’s time and energies— both d\iring and after 
regular school hours— is a factor that must be weighed in any de 
cision to undertake additional professional activities of an essen- 
tially elective nature. 

Although it is ultimately impossible to make a fully compelling 
case for research involvement on the part of the foreign-language 
teacher, it is possible to identify certain benefits of both a personal 
and professional nature that could be expected m result from a 
modest degree of attention to research and research concerns. 
First, a working familiarity with the p\irposes, techniques, and 
outcomes of ianguage-teachii^ research would help to moderate 
the highly polarized opinions about research and research workers 
held by many in the teachir^ field. At one extreme is the teacher 
who accords the status of pedagogical gospel to any “research 
fin ding ”; at the Other extreme is the teacher for whom all experi- 
mental studies are meaningless exercises carried out by “research 
people” far removed from classroom realities. One goal, then, of 
the teacher’s research involvement would be to gain a more bal- 
anced view of research as a field of inquiry that exhibits both suc- 
cesses and failures. With this more moderate view would come 
the ability to judge the significance or lack of significance of a 
pai'ticrilar research study, and to assess its contribution to peda- 
gogical theory and practice* 

A second benefit, closely related to the first, would be the 
avoidance or minimization of the bandwagon partisanship that has 
typically accompanied publication of such “bombshell” research 
studies as the Keating report on language-laboratory use, or more 
recently the Pennsylvania si!:udy comparing audiolingual and tradi- 
tional teaching methods.^ In these and similar instances, many of 
the unproductive charges and countercharges, debates andpoleinics 
stimulated by the reports might have been avoided if a greater 



^ Although for convenience the single term “teacher” will be used throughout 
the report, these discussions are intendedalso for language coordinators, super- 
visors. prospective language teachers, and others who areinterB^rted-jja^foreign- 
language research but who have little or no formal experience in this area* 

^ Keating (1963). 

3 Smith and Berger (1968); Smith and Baranyi (1968). 



proportion of teachers and other members of the profession had 
carefully read the reports in question and had arrived at an inde- 
pendent, reasoned appraisal of their shortcomings, merits, and 
practical significance. It is the rare research study that is all good 
or all bad. More typically, a given investigation has certain flaws 
but at the same time has something to say; and to the extent that 
language teachers and others in the field can learn to avoid an 
oversimplified, black-and-white approach to resesirch studies, the 
accurate evaluation of those studies and the application of any per- 
tinent and reliable findings to the actual business of classroom 
instruction will be facilitated. 

A third benefit inherent in the teacher’s research involvement 
is that he becomes aware of and has easy access to a large body 
of accumulated knowledge in diverse areas of foreign-language 
instruction. This information can be of considerable value to the 
teacher who wishes to try new approaches in his own classroom but 
who seeks assurance that these approaches have been found usef-al 
for the intended purpose through controlled experimentation. 

A significant general benefit of an increased involvement of 
foreign-language teachers in research is the growth in status of 
foreign- language teachii^ as a well-conceived, rationally sup- 
ported, and thoroughly professional endeavor. Rather than relying 
for research initiative on psychometricians, psychologists, and 
others who are, in general, only peripherally associated with 
foreign- language teaching, language teachers themsfdves must as- 
sume the major share of responsibility for initiating and carrying 
out research studies that are of greatest importance to them in 
their classroom role and, by the same token, of the greatest po- 
tential benefit to their students. 



Although these and perhaps a few other tangible benefits 
can be cited in support of the practicing teacher’s involvement in 
research concerns, the final and most personal justification for 
such an involvement must come from that indefinable motivation 
that interested language teachers have always had for going 
beyond mere job requirements into other areas of profes- 
sional development and activity. The same impetus that leads a 
teacher to create a “cultural island” in the classroom, to obtain 
and make use of realia, to sponsor language clubs or to show 
movies, to find, foreign pen pals for his students, to attend summer 
institutes and special training programs, to participate in language- 
teaching conferences and other professional meetings, must be the 
major impetus for his involvement in language-teaching research. 
While documents such as the present paper can and should suggest 
ways in which the teacher’s investment of time and energy in the 
research area can be most effective, the final decision to make 
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such an investment must rest with the teacher and must reflect 
his willingness to go beyond the minimum recjuirements of his 
daily teaching role. 

A Program for “Research Involvement” 

A suggested program to help the teacher become more familiar 
with and more closely involved in research includes thi*ee major 
areas of activity: (1) locating research, (2) evaluating research. 
(3) conducting research. 

“Locating research” involves as a preliminary step a certain 
amount of o verview reading intended to provide a concise and ac- 
curate survey of the various areas in which language— teaching 
research has been undertaken and to outline the broad theories of 
language learning that have given (or that should give) direction to 
instructional research. In the course of this overview reading, the 
teacher should also attempt to identify one or more specific re- 
search areas that are of particular interest to him and that would 
serve as investigatory threads to be followed in a topical reading 
program. It is at the topical reading stage that the teacher would 
learn to locate published research in a given area and to make 
effective use of language-teaching bibliographies and other in- 
formation sources. 

Simultaneously with the development of skills in locating re- 
search, the teacher would gain experience in evaluating research 
through the critical reading of experimental studies in an area of 
interest. The teacher’s own background knowledge of lan^age- 
teaching principles and procedures, together with a familiarity 
with certain basic research-reading techniques, should be-with 
practice-largely sufficient to arrive at an accurate appraisal of 
the validity and usefulness to the profession of a particular piece 
of research. 

A final, optional step would be the carrying out of a research 
project in some area of interest, either alone or in collaboration 
with other language teachers or professional staff. Although such 
an undertaking would certainly require considerable time and ef- 
fort, the prospect of conducting a stu^ of one’s own should have 
strong appeal to any teacher who in the course of his earlier in- 
volvement with the research literature has become interested in 
the closer investigation of a particular topic or phenomenon. The 
possibility of conducting experimental studies in the language- 
teaching area is by no means closed to persons lacking formal 
research training. Indeed, the knowledge-about-research gained 
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by the teacher over the first two stages of the suggested program 
would go far toward providing the necessary background. This 
background, together with any necessary assistance from psycho- 
metricians, technicians in statistical analysis, or other such per- 
sons usually available within the school system, should bring the 
conduct of at least modest research projects well within the pur- 
view of the interested foreign- language teacher. 



II. LOCATING RESEARCH 



Overview Reading 

The foreign- language teaching profession is fortunate in having 
available to it a number of comprehensive and well-written over- 
views of research undertakings that have been carried out in this 
field. In the Encyclopedia o f Educational Research (1960), Birk- 
rnaier reviewed the research literature published between 1940 
and 1960. Carroll, in a later edition of the same work (1969), 
summarized studies conducted up until quite recently. Other im- 
portant reviews of the research literature are available.^ Of 
particular importance for overview purposes is the extensive 
annotated bibliography of foreign-language-teaching research com- 
piled by Nostrand that covers the years 1945-64. Useful sources 
discussing basic theoretical considerations in foreign- language 
pedagogy and related research are also available.^ 

It is not intended that the teacher carefully and exhaustively 
peruse all of these sources at the outset. It would be hoped, how- 
ever, that by reading through these materials the teacher would 
get a general notion of the different areas of language pedagogy in 
which research has been conducted over the past twenty or thirty 
years, and conversely, those areas in which considerable investi- 
gatory work ha? yet to be done. 



In addition to providing background information, an important 
goal of the overview reading would be to help the teacher identify 
one or two language-learning topics or problem areas that he 
would like to examine in considerably greater detail. The topic or 
topics selected for closer investigation could be drawn from a 
wide number of categories, including for example: the teaching of 
foreign languages to young children, the relationship between lan- 
guage aptitude and learning ability, the role of student motivation 



^ Birkmaier (1958); Birkmaier and Lange (1967); Carroll (1966), 

^ Carroll (1960, 1965; overviews); Charles A, Ferguson (1966; applied lin- 
guistics); Ornstein (1968; programmed instruction; Rivers (1964; psychology 
of language learning), g 
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in classroom learning, programmed instructional techniques and 
materials, test construction theory and the development of valid 
and efficient language- testing instruments, optimum procedures 
for introducing and reinforcing vocabulary items, linguistic and 
pedagogical considerations involved in taking students beyondbasic 
dialogires and pattern sentences into a freer and more active use 
of the language, and so forth- In all cases, the primary considera- 
tion is the degree to which the topic interests and is of potential 
value to the teacher. 



Topical Reading 

His specific research area identified, the teacher should return 
to the overview studies previously described to note relevant 
works in print. While it is not necessary or even realistically 
possible for the teacher to read all of the previous studies on a 
given topic, it is important for a balancedview of the problem area 
and as a background for interpreting current research in the area 
that the most significant prior studies be examined in some detail. 
Commentary by the authors of the overview articles cited will be 
helpful in identifying the important works in the area, and summary 
descriptions of individual studies in Nostrand or other annotated 
bibliographies should also help to differentiate *^must read^ studies 
from the more limited or peripheral investigations. 

It is entirely understandable that persons with little prior ex- 
p0pj0j^Q0 vn research and the evaluation of research studies would 
initially have a certain amount of difficulty in reading typical re- 
search articles or other experimental reports. Nonetheless, it is 
possible— with good will and reasonable perseverance, together with 
the knov/ledge of a few basic principles and techniques of “research 
reading’'-for the novice to acquire quite rapidly the ability to 
read research documents with insight into their adequacy, accur- 
acy, and overall significance. The guidelines shown in the later 
section on evaluating research should be of help in the initial 
stages, and increasing facility can be expected after the first two 
or three reports have been read with some degree of application. 

Following or in conjunction with a reading of important previous 
work in a chosen research area, the teacher would want to adopt a 
system for identifying, on a continuing basis, current literature in 
the field of interest. An important initial suggestion would be for 
him to subscribe, if he does not already do so, to certain of the 
most important journals in the language-teaching field. Although 
it is possible in theory to read someone else’s copy of a given 
journal, or to review periodically the school library’s copy, it is 
usually difficult to do so in a convenient or consistent way. Further, 
the possibility of making marginal notes on the documents them- 
selves or of keeping an at-home collection of earlier issues would 
imply a personal subscription^. 



7 



There are several language journals to which one might sub- 
scribe.^ Certainly, both Foreign Language Annals and The Modern 
Language Journal would be at or near the head of any list of 
periodicals covering research on language- teaching methodology. 
In addition to presenting research articles as such, Foreign Lan- 
OTage Annals fulfills importaiit bibliographical functions that will 
be described latero 

In addition to these two journals, the teacher interested in for- 
eign-language research would v/ant to select other publications on 
the basis of his own language speciality or particular’ interests. 
The journals of the AAT’s would of course merit early considera- 
tion ( French Review, German Quarterly. Hlspania, Italica , Slavic 
and East European Journal ). Two recently established periodicals. 
Die Unterrichtspraxis and the NALLD (National Association of 
Language Laboratory Directors) Journal, report general peda- 
gogical studies and language laboratory/ audiovisual studies, re- 
spectively. Language Learning, published by the Research Club 
in Language Learning at the University of Michigan, contains both 
deBcriptive articles in a variety of linguistic areas and language- 
teaching research reports. 

Other journals worthy of mention are Le Fran«gais dans Is 
Monde, covering primarily the teaching of French, and the Inte r- 
national R e view of Applied Linguistics ( IRAL ), devoted to peda- 
gogically oriented linguistic studies and to more general psycho- 
linguistic topics. The publications of the American Educational 
Research Association (AERA)— which include the American Educa - 
tional Research Journal, the Revi ew of Educational Research, and 
8 newsletter. Educational Researcher~ are concerned with research 
in the field of educ ation generally. However, they often report 
studies of relevance to foreign- language teaching. Membership in 
the AERA includes a subscription to these periodicals. 

® Subscriptions to most of these journals also incJude membership in the cor- 
respoiiding professional organization-a worthwhile benefit in its own right. For 
subscriptions to the journals cited, contact: Foreign Language Annals: American 
council on the Teaching of Foreign Languages, 62 Fifth Ave., New York, N.Y. 
10011; Modern Language Journal: Wallace G. Klein, Business Manager, 13149 
Cannes Dr., St. Louis, Mo. 63141; French Review; F.W. Nachtmann, Executive- 
Secretary, American Association of Teachers of French, 59 East Armory Aye., 
Champaign, 111. 61820; German Quarterly and Die Unterrichtspraxis: American 
Association of Teachers of German, 339 Walnut St., Philadelphia, Pa. 19106; 
HispftTiiflr Eugene Savaiano, Secretary- Treasurer, Americ^ Association of 
Teachers of Spanish and Portuguese, Wichita State Univ., Wichita, Kan. 67208; 
Italica: Joseph E. Laggini, Secretary- Treasurer, American Association of 

Teachers of Italian, Rutgers Univ., New Brunswick, N.J. 08903; Slavic and 
East European Journal: Univ. of Wisconsin Press, Journals Dept., Box 1379, 
Madison. Wis. 53701: National Association of Language Laboratory Directors 
Journal: Sister Dorothy Zimmerman, Treasurer, NALLD, Mount Mary Coll., 
Milwaukee, Wis. 53222; Language Learning: Language Learning, 2001 North 
University Bldg., Univ. of Michigan, Ann Arbor, Mich. 48104; Le Frangais dans 
le Monde: European Publishers Representatives, 36 West 61st St., JMew York, 
N.Y. 10023; International Review of Applied Linguistics: Center for Curriculum 
Development, 401 Walnut Street, Philadelphia, Pa. 19106; Review of Educational. 
Research, ATn erica n Educational Research Journal, and Educational Re ^archer: 
Richard” A. Dershimer, Executive Officer, American Educational Research 
Associ 2 .tion, 1126 16th Street, N.W., Washington, D.C. 20036. 

. . f 8 9 



o 

ERIC 



erusal of articles appearing in a limited number 
,jf i»jMri Is will greatly assist the teacher to keep up with 

iurfent ^re in a particular area. However, this activity 

<u|i lemented by a monitoring system that will allow 
in« tH pick up potentially interesting references from 

lan the journals regularly consulted. Of consider- 
*/,lr in this task are the facilities provided by the var- 

i.jui* fcMlt itidi ational Resources Information Center) Clearing- 
mmmcs tar tk* ments in education and related disciplines, funded 
Try tiwr t «mi e of Education. Of particular interest to the foreign 
i*nr**<* ’ is the Clearinghouse on Languages and Linguis- 

ti. «, y the Modern Language Association of America, 

tht* .. se, usually referred to as MLA/ERIC, routinely 

le ERIC system research reports and other docu- 
3 the teaching of all foreig^ languages (Including 
» •» • •• 'ond language), the preparation of foreign-language 

iri*« Arr». ih* 1' ychology of language learning, presentation of the 

*tu» iAl> rcultural content, psycholinguistics, theoretical and 
s, and bilingualism. 



tmr fi ikjtuffii ut is reviewed, catalogued, and abstracted. These 
*rr hen indexed and cross-referenced according to 
ive terms and reproduced, along with author, title, 
•jul mokttvie iftl rmation, in the monthly publication, Research in 
Thus, by scanning current issues of RIE according 
Dical categories, the user can identify appropriate 
•:hk I documents and get some idea of the scope and 

• at le document by reading the corresponding abstract. 

\ oduction service is also part of the ERIC system, 

•Atf '-rf ill erials referenced in RIE canbe obtained in either 

ol t«u licrofiche or hard copy, from the Eric Document 

: -rvice (EDRS), P.O. Drawer O, Bethesda, Md. 

3o«>t4. 5 is quite inexpensive, but must be read with a spe- 

c 1*1 *' ice. Hard copy is somewhat more ejipensive but can 

tm tutatOma wl read in the same way as ordinary printed matter. 



Ifelul* Nit It the basic source of information for all ERlC-pro- 
I ■ ■■■it useful lists referencing all documents in for eign- 

t f C> a available through the reproduction service are 

rtmciiMtt fed in Foreign Lang uage Annals and merit con- 

•> overview of recent puDiicanons in the lai^^e- 
toaduan Although these lists do not include abstracts, they 

•iWMi »• nt*** ce numbers by which the appropriate abstract nmy 
I* The C urrent Index to Journals in Education ( CIJE ) 

jy list in g and semi-annual and annual cumulations 
at attacrnttam-ri' g^ted articles appearing in sociological, psycho- 

other journals as well as those published 
t* tm r » 1 il W «■ ucational journals. MLA/ERIC currently monitors 
' -xty-five journals that publish articles on foreign- 
- and linguistics. 



A ran IN^' gnt source of bibliographical information is the 
MLA/nilC/At' Annual Bibliography on the Teaching of Foreign 




ublished so far appear in the December 1968, March 1969, 
h 1970, October 1970, March 1971, and October 1971 issues 
i each list are available from MLA/ERIC, and microfiche 
ductions cmi be obtained ftofiiEDRS. 
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Laneuaees. This „ bibliography, routinely published m Foreign 
Lan^^e Annals,® is based on a continuing review of almost three 
hun^^ journals, including both language-learning journals and 
journals primarily devoted to psychology, sociology, or other ^s- 
ciplines but which occasionally print articles of interest to foreign- 
language teachers. The Annual Bibliography has an indexing and 
cross-referencing system that facilitates locating references in a 
specific area of interest. The bibliography does not, however, con- 
tain abstracts, and it is thus necessary to refer to annotated sources 
such as RIE or to the original document to obtain information about 
the document beyond that supplied by its title and indexing in the 
bibliography. 

A number of other bibliographical sources might be mentioned, 
including especially Dissertation Abstracts, covering theses and 
dissertations written at U.S. universities in all subject areas, and 
Psychologi cal Abstracts, devoted primarily to topics in psychology 
an d allied fields but occasionally reporting studies having applica- 
tion to foreign- language pedagogy. 

Additional bibliographical sources will become apparent and 
appropriate in the course of the teacher’s developing interest and 
background in a particular research area. It is suggested, how- 
ever, that the teacher begin his literature-monitoring program on 
the basis of at-home subscriptions to a few carefully selected 
journals, supplemented by the monthly or annual perusal, as 
propriate, of entries in the Lists of ERIC Documents,. C^, 

and the MLVERIC/ACTFL Annual Bibliography. 



III. EVALUATING RESEARCH 

It is unfortunate that the term “research report* conjures in the 
mind of many teachers an arcane, cabalistic document that can be 
understood only by a select group of persons who have undergone 
intensive training in research methodology. While a formal back- 
ground of this sort is never a hindrance in the accurate ?jid com- 
prehensive appraisal of research documents, persons with little or 
no prior training in research can-by making use of their own ex- 
tensive knowledge of the foreign-language-teaching field and by 
following certain basic research-reading guidelines-determme with 

surprising accuracy the overall validity aud usefulness to the 
profession of a given piece of research. The next few paragraphs 
describe in some detail what should be looked for in the critical 
reading and evaluation of a typical research report. 
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For 1967, the Aimual Bibliography was published in four successive Issues 
of FLA, beginning in October 1967. A merged edition is available from the ERIC 
Document Reproduction Service. The 1968 Bibliography was published as a single 
work in the May 1969 issue of FLAj the 1969 Bibliography, in the May 1970 
issue; and the 1970 Bibliography, in the May 1971 issue. 
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Is it really a research study? 



Many of the articles published in language-teaching jo^nals do 

not-nor do they intend to-report fornaal research studies. The 
term “research study” should be applied only to certain informa- 
tion-eathering activities that conform to specific and highly objec- 
tified procedural rules, and that are subject to evaluation and 
criticism by others on the basis of their conformance or lack of 
conformance to these rules. In keeping with the objective nature of 
the research activity itself is the high degree of detachment and 
succinctness with which research documents are generally 
pared. A typical research report consists of a highly factual de- 
scription of; (1) the research question or questions at issue (the 
hypotheses to be tested); (2) the nature of the ei^erimental group 
(characteristics of the students or other participants in the ex- 
periment); (3) the eiqierimental procedures used; (4) the tests 
administered or other measurements taken to determine the out- 
come of the eiqieriment; and (5) the nature and results of any 
statistical or other kinds of analysis carried out. Only at the end 
of the report, in a conclusion or discussion section, v/ould there be 
any element of personal opinion; and even here, general or sub- 
jective statements by the author would be minimal and closely tied 
in with the formal, objective results of the study. 



Nonresearch articles take a wide variety of forms, ^d are 
properly evaluated according to such criteria as the reputation and 
background of the author in the area under study and the inherent 
logic and validity of the arguments presented. It is certainly not 
intended to suggest that nonresearch documents are lacking in 
usefulness to the profession or that they play an inconsiderable role 
in scholarly endeavors. Rather, the purpose of distinguishing be- 
tween research and nonresearch studies is simply to emphasize 
that different principles of evaluation apply in the two cases. 

Once it has been determined that a particular article does in- 
tend to describe a true research stu^, the reader is then m a 
position to apply to that article a number of review procedures that 
have been generally adopted for the evaluation of research studies 
and that are powerful and wide-ranging in that respect. Although i 
is not possible to list the many specific questions that might be 
asked of a given research study, it is possible to characteri^ 
the basic types of questions that would beat issue in a wide variety 
of research undertakings and that would form the nucleus of a 
reasoned and comprehensive appraisal of these undertakings. 



Does the study address a significant problem in langu age-teaching 
methodology ? 

There is a wide difference among research projects in the 
number and scope of the problems or questions investigated. At 
one extreme is the small-scale, carefully delineated study of a 
single phenomenon such as the student’ s learning of foreign-lanp^e 
speech sounds under carefully controlled conditions of presentation 
and practice. At the other extreme are the more global studies of- 
entire language-teacj^^. methodologies, such as the comparison 

li 



of audiolingual and traditional methods carried out in the Pennsyl- 
vania study* Regardless of the scope of a particular study, the 
reader must ask himself whether genuine educational problems are 
being investigated-that is, whether information gathered as a re- 
sult of the study would in fact have some useful application in 
real-life teaching situations. For those studies carried out in 
school settings and involving an investigation or comparison of 
clearly defined instructional procedures, real-life applicability is 
usually quite apparent. Studies of amore restricted or “laboratory” 
nature may of course also have obvious classroom significance, 
but in many cases the connection is not so readily apparent. Thus, 
except for investigations that are clearly addressed to “basic- 
research” topics, 9 the author of ar^ortbased on experimentation 
in a nonclassroom situation should be expected to describe in at 
least general terms how the results of his study might be trans- 
lated into teaching practice. 

Do es the study take into account important prior research? 

If the teacher has followed the background-reading suggestions 
made above, he should have some acquaintance with earlier basic 
studies in the area tinder investigation. The author’s eiqilicit 
mention of such studies would be expected, usually in the intro- 
ductory paragraphs. While the presence of a large number of 
references is by no means an infallible indication of the overall 
quality of the report, the absenceof references to previous closely- 
related studies would raise the possibility that the author does not 
have a comprehensive background in the field. 

Are the important chgiracteristics of the student participants or 
other experimental subjects clearly and completely described^ 

Such basic data as students’^^ age, sex, grade level, nature and 
extent of previous lani^uage training, type of school attended, and 
very importantly, the basis on which they were selected for the 
study (e.g., all the members of Miss Jones’s class, every third 
student from a listing of the entire school grade in that language, 
any student who was willing to volunteer for the study, etc.) should 
be included. Detailed information about the students participating 
in the study is necessary both to evaluate the reported outcomes 
of the study and to judge the extent to which the findings could be 
extrapolated to other groups of students. The possibility of extrap- 
olating or generalizing from the experimental group and situation 
to other groups and situations is one of the primary motivations 
for conducting research studies, and this aspect will be discussed 



For a discussion of important distinctions between “basic** and “applied** 
forms of educational research, see Carroll (1968). 

A technical term for an experimental participant is “subject,** often abbre- 
viated “S** in the research literature. However, since most subjects in educa- 
tional experimentation are in fact students, it is appropriate to use the more 
familiar term here. 
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in greater detail in later paragraphs. The point to be made here 
is simply that extensive description of the status and background 
of the experimental participants is required if the reader is to 
obtain any idea of the extent to which such extrapolation is possible. 



Is the experimental design sufficiently rigorous to rule out con- 
founding factors? 

Many investigat’ons generally considered to be true research 
studies are pseudoresearch studies in that the ejqjerimental design 
does not include all of the procedural steps necessary to rule out 
the possibility that some factor or factors other than t he ejqjeri— 
mental treatment was in fact responsible for the observed re- 
sults. Barnes (1964) lists four types of “rese^ch” desip com- 
monly encountered in educational ejqjerimentation, of wnich only 
the last incorporates all of the important procedural elements: 
(1) “after-only” study without control group, (2) “before-after” 
study without control group, (3) “after-only” study with control 
group, (4) “before-after” study with control group. “Before” and 
“after” indicate the administration of appropriate tests or othp 
evaluation instruments prior to or at the completion of the experi- 
ment. “With control group” and “without control group” denote the 
presence or absence of an additional poup of student participants 
who are closely similar to the experimental group in background 
and prior training and who participate in all of the activities of the 
ejqjerimental group except for the single ejqjerimental activity or 
procedure being investigated. 

In order to point out the differences among these various de- 
signs, let us assume that a classroom teacher has prepared a 
series of tape-recorded conversations and other passages in the 
target language that are carefully arranged in order of increasing 
lexical and grammatical complexity and speed of delivery. The 
research question is the extent to which student practice in listen- 
ing to these materials in the languap laboratory will increase 
general listening-comprehension proficiency as measured by a 
standardized test. In an “after-only” study without control group, 
the teacher would simply work with one group of students (the ex- 
perimental group) and would testtheir listening-comprehension pro- 
ficiency at only one point in time— following the ejqjerimental treat- 
ment of listening practice. Although this type of study has the ad- 
vantages of speed and simplicity, it has major drawbacks that 
soriously limit its usefulness and validity as a true ejqjerimental 
procedure. In the absence of information on the students’ listening- 
comprehension proficiency prior to the e^qJeriment, it could be 
suggested that the laboratory listening practice had little or no 
beneficial effect but rather that the end-of-ejq)eriment listening 
test scores reflected only the students’ initial proficiency level, 
that is, the listening proficiency they had acquired from prior 
classwork or other training. 
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Possible influences arising from this prior study or ejqjosime 
could, however, be controlled by means of a “before-after” design 
in which student performance would be tested both prior to and at 
the end of the experiment. By comparing, through appropriate 
statistical procedures, the test scores obtained at the end of the 
e?q)eriment to those obtained at the beginning, it would be possible 
to establish a certain direction and extent of change in listening 
proficiency (hopefully, positive and large) over the time period of 
the experiment. The results of a “before- after” study conducted 
with a single group could be considered a valid reflection of the 
merit of the experimental procedure provided that no Influence 
other than procedure itself could reasonably be suggested to ac- 
cbunt for the “before- Editor " difference in student performance. 
Unfortunately, there are usually a large number of alternative 
explanations that can be proposed for the results of “before-^ter" 
studies using a single experimental group. A reader of this re- 
port might legitimately suggest that the students’ listening-com- 
prehension level was not increased as a result of the practice 
sessions in the laboratory but, rather, as a result of the normal 
classroom listening experiences that the students had over the 
same time period. 

In order to rule out possible contamination of the observed re- 
sults by these or other external factors, an additional experimental 
safeguard would be required: the use of a “control" group that woidd 
participate in all activities and undergo all influences of the ex- 
perimental group except for the specific training procedure under 
inve stigation. In terms of the example experiment, the teacher 
would at the beginning of the study establish two separate groups 
of students: an “ejqperimental” and a “control” group. The two groups 
would be set up either by matching students on the basis of years- 
of-instruction, class grades, and other background variables an- 
ticipated to affect their performance on the experiment or (more 
conveniently and probably more appropriately) by assigning the 
students to the two groups on a strictly random basis. The teaching 
situation would then be arranged so that both the experimental and 
control groups would have the same classroom experiences and 
would be treated similarly in all respects, except that the experi- 
mental group alone would receive the laboratory-listening 
practice.il 

Again, an analysis of results would be made by comparing “be- 
fore" and “after" scores for the experimental group, but with the 
important difference that corresponding “before" and “after" 
scores would also be available for the control group. If koth the 
experimental and control groups showed comparable increases in 



So that total instructional time would remain the same for both griups, the 
teacher would probably arrange for the control group to carry out some unre- 
lated activity such as reading a short story or other material during the period 
that the experimental group was^^lwing to the practice tapes. 
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performance, it would be doubtful that the laboratory practice 
per se had any beneficial results. If, however, the increase in test 
score of the e:q)erimental group was much higher th^ that of the 
control group, a strong case could be made for attributing these 
gains to the laboratory-listening practice since-thanks to the ori- 
ginal e3q)erimental design-presence or absence of the listening 
practice had been isolated as the only substantial difference be- 
tween the two groups. 

How, then, should the critical reader evaluate the results of a 
research study exemplifying one or another of these experimental 
designs? Only those e3q)eriments that operationally rule out the 
possibility of confounding influence by other-than-ejqjerimental 
factor s-including both prior student knowledge and peripheral in- 
fluences in the course of the e3q)eriment— should be considered true 
research studies in which a considerable degree of confidence 
could be placed. Because it is often difficult to set up a tight ex- 
perimental design embodying all of the desired controls, many of 
the ejqjeriments reported in the language-teaching literatirre have 
been based on less than optimum designs. Studies of this fype 
should not be dismissed out of hand but should be carefully read 
in an attempt to estimate the probable type, extent, and direction 
of nonexperimental influences that were operating in each indivi- 
dual case. Although studies that employ a less rigorous design 
cannot be considered definitive investigations, they may often be 
of considerable value in suggesting trends or pointing out useful 
areas for further experimentation. 

Are the tests or other evaluation procedures appropriate for the 
Intended purpose, and are they correctly used and interpreted? 

All research studies, of whatever fype, involve the collection of 
data on student performance through the use of some evaluation 
instrument or procedure. These instruments or procedures typi- 
cally take such forms as the classroom observation of student 
behavior, the use of various questionnaires, or the administration 
of standardized or locally-prepared tests. In all cases, the validity 
and significance of the study hinge on the appropriateness of these 
measurement techniques and on their accurate utilization. 

Of the various evaluation procedures, classroom observation 
is probably the most difficult to carry out in an objective maimer. 
Researchers who make use of this technique must be very careful 
to adopt an observational system that minimizes subjective judg- 
ments on the part of the observer and that lends itself to reliable 
tabulation of the observations made. A procedure known as “inter- 
action analysis”12 has shown promise for objectively describing 
the behavior of both students and teachers in the classroom situa- 




For further information on this technique see Medley and Mitzel (1963), and 
Moskowitz (1967). . 
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tion. However, these techniques are by no means so highly re- 
fined or well understood as are other evaluation techniques, 
this reason, the reader of a research study based on classroom 
observation should look for, and should expect to find, a detailed 
description of the observational system used and the manner in 
which the final results were tabulated. 

For research studies making use of questionnaires, a compre- 
hensive report of evaluation procedures would include a detailed 
description of the questionnaire, including a number of sample 
items from the questionnaire or reproduction of the entire ques- 
tionnaire. Since a reasonably large number of replies to a given 
question are needed for reliability, questionnaire results should 
show the number of students responding to each question, and not 
merely percentage figures. 

Questionnaire data may influenced by many subtle factors 
including the degree of student motivation in filling out the ques- 
tionnaire, the format, wording, and sequencing of questions, and so 
forth. Payne (1951) provides excellent discussions of questionnaire 
design., use, and interpretation. 

The administration of a test or battery of tests is the data- 
collection procedure used in thegreatmajority of language-teaching 
research studies. In all cases, the most important question is the 
extent to which the content of the test corresponds to the behavior 
the researcher is attempting to measure. For example, let^us 
assume that an investigator wishes to measure the accuracy of 
students* pronunciation of certain sounds when speaking the for- 
eign language in a conversational situation. The testing procedure, 
however, involves only the student mimicry of a tape-recorded 
master voice. Thus, through a serious incongruity between the 
kind of student performance tested by the evaluative instrument 
(imitation of a model sound) and the performance that the investi- 
gator had intended to measure (accurate student-initiated pronun- 
ciation in an untutored situation), the results of the study with re- 
spect to the stated research goal are open to serious qfuestion. 

By what means can the relatively unspecialized reader of a re- 
search study evaluate the suitability of the tests used for the 
purpose intended? 

First, the author of the report must provide sufficient information 
about the tests administered. In the case of standardized, mxiltiple- 
choice tests, minimal information would include the name of the 
test, the publisher, and the test form and level. 

For standardized tests that are not multiple-choice (these would 
include primarily tape-recorded speaking tests and “fill-in** 
or “short-essay” types of writing tests), detailed information 
on the scoring procedures used should also be provided, 
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including information on any steps taken to determine scoring 
reliability, such as the rescoring of individual tests by the same 
rater or the pooled scoring of each test by a number of raters. 

For many research applications, appropriate commercially dis- 
tributed tests are not available, and the researcher must design 
his own instruments. In describing locally prepared tests, the 
author of a comprehensive research report should give much more 
information than that required for standardized tests. At a mini- 
mum, this would include: (1) identification of the corpus of ma- 
terials on which the test questions are based (e.g., the vocabu’.ary 
or grammai’ appearing in a certain textbook or textbook section; 
random or systematic selection of items from a frequency list or 
other tabulation); (2) a description of the question format (mul- 
tiple-choice, fill-in-the-blank, written responses to spoken ma- 
terial, etc.); (3) a statement of the total number of items in the 
test arid the amount of time allotted the student; (4) a description 
of the scoring procedures used, including information cn any re- 
liability checks conducted. Finally, several sample items should 
be presented. 

Even if the author has provided a complete and accurate de- 
scription of the tests used, it remains the responsibility of the 
reader to evaluate the appropriateness of the tests for the intended 
purpose. Tnere is, unfortunately, no prac'lcal way to acquire this 
ability other than by becoming familiar with the nature and content 
of the major published foreign-language tests and by acquiring a 
reasonably extensive fund of information about language testing 
in general. 

For published tests, the appropriate test catalog, sample test 
materials, test manual, and norms tables should be obtained from 
the publisher and read as basic background information. This is 
not a difficult undertaking, since there are relatively few published 
tests or test batteries in common use in foreign-language research 
studies. The detailed test reviews appearing in the series of 
Mental Measurement Yearbooks^ ^ should also be consulted. How- 
ever, a reading of these reviews should not take the place of a 
close examination of the t3sts, their manuals, and other supporting 
materials. 

Developing an ability to judge the appropriateness and technical 
merit of nonstandardized or locally prepared tests is by no means 
beyond the scope of the interested teacher. Useful background 
works in thisrespect include those by Lado (1964) and Valette (1967). 
Lado presents a theoretically oriented overview of language-testing 
procedures, with particular emphasis, on evaluation concepts de- 



Buros (1965); see also earlier volumes by the same editor. 
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rived from contrastive linguistics. Valette outlines and discusses 
many different types of tests and test questions that can be devel- 
oped on a local basis and gives practical suggestions on their 
appropriate use and interpretation. A testing work-kit developed 
by the Educational Testing Service^^ discusses the selection and 
use of standardized tests and also provides useful guidelines for 
local test development, scoring, and interpretation. 

Is the statistical analysis appropriate to the data gathered and 
is it properly carried out? 

Every experimental study recjuires some type of statistical anal- 
ysis of the information obtained. The purposes of this analysis 
are: first, to translate the e3q)erimental data from their original 
raw form (e.g., entries on a classroom-observation form, ques- 
tionnaire responses, individual test scores) into a more compact, 
more easily interpretable summary form; and second, to determine 
by means of specific computational tests whether the results of 
the experiment are statistically significant. The term “significant,” 
as used in the statistical context, has a very precise meaning that 
should not be confused with ordinary or general connotations. Re- 
sults of an experimental study are considered statistically signifi- 
cant when the experimental data are found to be of a sufficient 
overall magnitude (or, in comparative studies, sufficiently different 
from the “control” group data? that there is only a small probabil- 
ity that the observed results could have been obtained by ch^ce 
rather than by the operation of the experimental procedure. Statisti- 
cal analysis permits the probability of ch^ce results to be deter- 
mined numerically. Thus, when an investigator states that his ex- 
perimental results are “significant at the o05 level,” he is saying that 
his statistical analysis indicates that there is only a five percent 
probability (or less) that the results of the experiment c^. be at- 
tributed to the operation of chance factors. In most educational re- 
search, '^ne cf the two probability levels, .05 and .01, is adopted by 
the researcher as the level he will accept as “significant”; by 
choosing one of these levels he indicates that he is willing to 
accept a one-in-twenty or a one-in-a-hundred risk that the results 
he reports as “significant” are in fact attributable to chance. 

Even if the results of a study attain statistical significance at a 
given level, this is by no means a guarantee that the results are 
significant in broader pedagogical terms. Suppose that in the labor- 
atory-listening practice experiment previously described the exc- 
perimental group was found to have “before-after” differences in 
scores on a fifty-item listening test that were about two pcints 
higher on the average than the control-group scores. Statistical 
analysis reveals that this difference is significant at the .05 level, 
but the question remains as to whether this relatively small average 
improvement in scores would justify the effort and expense re- 
quired to incorporate the practice-listening materials into the class 
or school language program. 

Tests and Measurements Kit; available from Jhfe Educational Testing Service, 
Princeton. N. J. 08540. 
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Thus, the reader of a research report should-in addition to 
checking for statistical significance of the results-always attempt 
to judge whether the absolute magnitude of the reported scores or 
score differences would be sufficient to make a genuine real- 
life case" for the experimental procedure ^mder examination. 

Although persons ^mtrained in statistical-analysis procedures 
can and should acquire an appreciation of the concept of statistical 
significance anH its application in the evaluation of research re- 
ports, it is doubtful that the self-study of statistical textbooks or 
other instructional materials will be of much use to the average 
teacher in gaining a close working knowledge of statistical pro- 
cedures and their application in a given research situation. A 
formal course in educational or psychological statistics would 
probably be required for this purpose, and it is suggested that the 
interested teacher take such a course during the summer or through 
extension study. 15 

In the absence of experience gained from formal course work in 
statistics, the teacher would probably have to enlist the help of a 
qualified person within the school sj^stem to evaluate the appropri- 
ateness of a given statistical technique as used in a particular re- 
search study. This is not to suggest that every research study 
read by the teacher should be subjected to such a review. Rather, 
since any research report must pass muster in a large number of 
nonstatistic ai respects before the question of statistical accuracy 
becomes appropriate, a close review of statistical characteristics 
could be reserved for those studies that are found to incorporate 
successfully all of the other characteristics of a valid and mean- 
ingful study. In this coimection, it :i;hould be emphasized that even 
the most complex and sophisticated statistical procedures caimot 
be considered to redeem in any way a study flawed by overall peda- 
gogical insignificance, inappropriate selection of student partici- 
pants, insufficiences of e3q>erimental design, or invalid measure- 
ment techniques. 

Is the study so designed as to allow at least some generalization of 
the r esults to other student groups andeducational settings, and are 
the stated conclusions in keeping with this level of generalization? 

Consider a hypothetical research report in which the author con- 
cludes that “due to the specialized characteristics of the experi- 
mental participar^ts and the specialized nature of the experimental 
procedure, it is not possible to extrapolate the results of this study 
to any other group of students or to any other curricular situation.” 
V/e would wonder why the study was conducted in the first place, 
since its results are considered nontranslatable to any other stu- 
dents or school systems. Although this example is clearly over- 



For those who would like to undertake the reading of basic statistics texts 
on ai “at-home” basi§, George A. Ferguson (1966) and Edwojfds (1967) are 
recommended as standard,, co^^^ensive treatments. 
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drawn, it does point up the need for the researcher to make pro- 
vision-in the design and conduct of the experiment-for his results 
to be legimately extrapolated to some group or groups other than 
those involved in the study and to some pedagogical situation other 
than the particular context of the experiment. 

In designing an experiment, it is relatively easy to incorporate 
at least some degree of generalizability. For example, rather than 
using “the brightest students- or “the students in Miss Jones’s 
class,” the experimenter might, through random selection or other 
means, identify an experimental group that could be considered 
typical of “all second-year language students in the high school.” 
By choosing participants for the experiment on this broader and 
more heterogeneous basis, the researcher would operationally rule 
out the possibility that the obtained results would apply only to 
“the brightest students” or only to “Miss Jones’s students” rather 
than to a larger and more interesting groi:p. 

Just as it is important for the researcher to provide for and to 
claim some useful degree of generalizability for his results, it is 
also important for him to avoid overgeneralizing his findings be- 
yond the limits imposed by his esqperimental design. It would be 
inappropriate for the esqperimenter in the previous example to claim 
that his results were applicable to “second-year high school stu- 
dents in the United States,” because he did not in fact e3q)eriment on 
a representative national group of students h*ut only on a repre- 
sentative group of students at one high school. An appropriate 
summary statement for this esqperiment might be that the 
results are typical of students at thelocalhigh school, and possibly 
t5rpioal of students in other areas who have the same general I.Q. 
and socioeconomic status and who have undergone the same type 
of instruction at a comparable school. 

Although the characteristics and basis for selection of the ex- 
perimental group are usually the most important factors affecting 
the generalizability of results, the nature of the esqperimental 
procedure itself plays a substantial role. Small-scale esqperiments 
such as those involving the learning of certain speech soimds or 
lexical items cannot usually be extrapolated with confidence to 
dissimilar or broader contexts, although they may be highly defin- 
itive in the limited area surveyed. By the same token, results of 
experimental studies using self- instructional or programmed ma- 
terials may be applicable only to the materials involved, unless 
it can be shown that their content and organization are comparable 
to other types of texts or materials. It should be incumbent on the 
author of a research report to describe the course of instruction 
involved in his esqperiment in enough detail to allow the reader to 
estimate whether the results are necessarily closely bound to the 
particular materials or procedures used, or whether they could 
reasonably be generalized to other texts v i* techniques with which 
they share certain common elements. 
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IV. CONDUCTING RESEARCH 

t% wtJuUi je highly unrealistic to suggest that each and every 
«ri«ju»rtaiDfr ceacher should at some point in his career conduct a 
iotm^ ^search study in some area of foreign-language instruc- 
ikjr is not, however, unreasonable to e^ipect that some teachers 
iio have devoted considerable time and energy to locating and 
reading about the research activities of others would become in- 
terested in the prospect of carrying out research studies of their 
own. 

The amount of work and personal application involved in con- 
ducting a useful experiment on some aspect of foreign-language 
teaching should not be minimized, but it should be made clear that 
such an undertaking is by no means beyond the capabilities and 
available resources of the practicing teacher. 

Prior detailed study and analysis of a number of different re- 
search reports in an area of interest would be one of the most im- 
portant types of preparation that could be made for carrying out 
experimental research. To this should be added the reading of 
various background sources cited in earlier sections of this report, 
and close attention to the broad procedural suggestions given below. 



C hoose a limited but significant research question^ 

A common tendency among beginning researchers is to choose 
a research question that is too complex and has too many different 
facets to permit adequate experimental study. Although experienced 
investigators often use research designs that allow them to test 
more than one hypothesis in a single experiment, it is much safer 
and less confusing for the novice researcher to identify a single, 

4th ed., ed. Robert L. Ebel. et al. (New York: Macmillan. 1969). 
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carefully delimited student activity that he wishes to evaluate and 
to investigate the effects of that activity independently of all other 
aspects. It should also be determined at the outset that the re- 
search question can be operationally formulated in terms of student 
performance on a particular evaluation instrument or procedure. 

Ideally, the topic would be selected on the basis of prior reading 
in an area of interest and would involve some area of inquiry for 
which suitable research information is not available. Investigation 
of a still unresolved problem would generally be required if the 
teacher were considering eventual publication of his research, and 
most teachers would want some assurance-^:hrough prior study of 
literature in the area-that they were not devoting a large amount 
of time and effort to a research question that had been compre- 
hensively dealt with in earlier studies. 

Arran ge for the collaboration of any necessary resource persons. 

Although the selection of a research problem and the develop- 
ment of a proced?iral outline for the stucfy should be the responsi- 
bility of the language teacher as “principal investigator,” it will 
often be desirable or necessary for him to collaborate with other 
persons, particularly in regard to data collection and analysis pro- 
cedures. This collaboration should begin in the early stages of the 
project, so that the experimental design selected will permit the 
use of a recognized and appropriate statistical-analysis technique. 
It would also be highly useful for the beginning investigator to re- 
view his plans for the experiment withaperson trained in research 
design. In many cases, an e^erienced researcher can suggest 
slight changes in the e:q)erimental plan that will make the stucty 
easier to carry out or that will permit more useful interpretation 
of the results. 

Discussion of the intended stucfy with other foreign-language 
teachers would provide a beneficial dialogue, especially on the 
practicality of the study and its pedagogical significance. A collea^e 
might also agree to serve as co-worker and provide help in carrying 
out the experiment, administering and scoring tests, and so forth. 



Write out a detailed research plan and review it carefully for con- 
c eptual and operational flaws. 

E3q)erienced researchers can often bypass the paperwork in- 
herent in drawing up a detailed opr, rational plan for a given stucfy, 
especially if the study is to follow a common and straightforw^d 
research pattern. However, it is consideredessentialfor the novice 
investigator to describe on pf^er and at some length all of the 
procedural aspects of the stucfy, including information in the fol- 
lowing areas^ ( 1 .) a statement of the research question, cast in 
operational terms (in the listening practice e3q)eriment, for ex- 
ample, the appropriate operational hypothesis would be that the 





student group \mdergoing language laboratory practice with the 
experimental tapes would show significantly greater “before-after” 
increases in listening comprehension test scores than would the 
control group); (2) a description of the way in which the ej^eri- 
mental group or groups will be constituted (i«e«y through pairing* 
random selection from a larger population, etc.), together with an 
indication of the level of generalization to other groups that can be 
anticipated; (3) a close description of the measuring instruments 
to be used and the way in which the measurements are to be made 
and tabulated; (4) a detailed “schedule-of-events” showing the 
points at which the various ejqjerimental activities are to take 
place; and (5) a description of the statistical- analysis procedures 
to be used. 

A detailed plan of this type is necessary to assure the teacher 
that all major steps in the experiment have been considered in ad- 
vance and that no significant design or procedural problems exist. 
The various elements of the plan should be checked against the 
criteria given in the preceding section, against other research 
guidelines such as those provided in Suydam (1968) and Van Dalen 
(1958), and then reviewed by any collaborating resource persons. 



Carry ou t the experiment in accordance with the operational plan. 

There is no single set of recommendations that would suitably 
cover the procedural aspects of all the various studies that might 
be carried out. However, the ejqjeriment should be conducted in such 
a way that no extraneous variables affect the course of the ejqjeri- 
ment or, if such influences cannot be avoided, in such a way that they 
affect both experimental andcontrol groups comparably. Particul^- 
ly to be guarded against is the so-called Hawthorne effect in which 
the teacher, by his own incautious approach to the experiment (e.g., 
through a show of obvious enthusiasm for the “new experiment^ 
technique”), may motivate the students to do better at the ejqjeri- 
mental task than would otherwise have been the case. Zimny (1961) 
describes this and other nonejqjerimental influences that may 
arise in educational or psychological research. A careful reading 
of this work will help to alert the teacher to various biasing 
factors so that he can guard against them from the outset. A good 
basic rule for ejqjerimental studies conducted in a school setting 
would be for the teacher to attempt, insofar as possible, to have 
the experiment go unnoticed— that is, not to alert the students to 
the fact that something special is happening or that there is any 
departure from normal course activity. 

The teacher should also plan to keep a detailed written record 
of the ejqjeriment, especially if the study is to extend over a mat- 
ter of days or weeks. Such a log would note the day or time at 
which each element of the ejqjeriment was introduced, the names of 
students present, any ^musual occurrences, important observations 

24 

2'3 ” 



made by the teacher in the course of the experiment, and so forth. 
A careful record of this type can be of considerable help in the 
interpretation of experimental results and in the preparation of 
an experimental report. 



Evaluate the statistical and practical significance of the study. 



The teacher would be quite fortunate if the results of his study 
showed both statistical significance-as determined by an appropriate 
analysis procedure— and practical significance— as suggestedby large 
increases in student test scores or substantial improvement over 
the performance level of a control group. Results that are statis- 
tically significant but which represent relatively small student 
gains are also useful in that they suggest the researcher may be 
on the right track in his investigations. A close reexamination of 
the details of such an experiment might indicate ways in which the 
materials could be improved or the procedure streamlined to pro- 
duce more substantial increases in student performance. 

Experimental studies that do not attain statistical significance 
are not necessarily failures. Provided that the design and conduct 
of the experiment are known to be correct and free of external, 
biasing influences, a finding of no significant difference based on a 
very high probability of chance success (perhaps .30 or higher) is 
useful in that it gives a frank indication of what is probably a 
“blind alley” of investigation. Such results, although less psycho- 
logically rewarding than positive ones, contribute to research 
knov/ledge by leading the investigator away from an apparently 
xinprofitable line of inquiry toward a reorientation or reformulation 
that may have a greater chance of success. 

Studies that barely fail to meet accepted significance levels 
(i.e., with chance probabilities of around .10 or .15) are often 
difficult to interpret. It is not possible, of course, to claim that 
the experimental outcome is statistically or pedagogically mean- 
ingful, but, on the other hand, the statistical results do come close 
enough to commonly accepted probability levels to suggest that 
some factors other than pure chance were operating in the ex- 
periment. Probably the best interpretative approach to a situation 
of this type would be to look very closely at the materials and pro- 
cedures involved to judge whether any changes in the experiment 
could reasonably be expected to produce a considerable positive 
change in student performance. If, for example, the experimenter 
were to find on reexamination that a significant body of information 
had been inadvertently omitted from the student materials and that 
this had probably affected performance in a substantial way, the 
initial “almost significant” results could be considered' suggestive 
of more substantial results that might be obtained following ap- 
propriate changes in the experimental materials. If, on the other 
hand, there is little indication that a restructuring of the e>q)eri- 
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merit would improve the results considerably, it would probably be 
better to discoimt the study at hand and to turn to some other 
avenue of approach. 



Prepare a detailed report of the study for local information or for 
publication. 

Except for those studies that are definitely failures in that they 
show serious conceptual or procedural flaws, every research 
undertaking should have the preparation of a detailed report as its 
final step. Such a report would provide the teacher and other per- 
sons in his school or school system a comprehensive record of 
the research activities undertaken and their results. 

Studies that are felt to have been conducted with sufficient rigor 
and with appropriate attention to earlier studies in the area should 
also be submitted to a professional journal or other information 
source for possible publication.^® It should be emphasized in this 
connection that there are no perfect experiments, and that even the 
most significant published research embodies minor flaws to which 
the researcher frankly admits and for which he attempts to estimate 
the direction and extent of experimental influence. Such short- 
comings, if truly of a minor nature, should not rule out the pos- 
sibility of having a study published, and there is every good reason 
to submit a comprehensive report to an appropriate publication. 

Much of the necessary backgroimd work for writing a report 
will already have been done if the teacher has made the suggested 
written plan-of- attack before starting the experiment and has kept 
detailed notes in the course of the experiment. In preparing the 
report, the teacher would follow the content outline generally used 
for research reports, consisting (with minor variations) of: intro- 
duction, hypothesis, subjects, procedure, analysis, results, dis- 
cussion, and conclusion. He would be careful to include, in an ob- 
jective and succinct manner, ai. the information that the reader 
would require if he were to diplicate the experiment on his own. If 
all other aspects of the suggested “research involvement program** 
have been followedr-including the critical reading of a number of 
typical experimental reports-the teacher should find the prepara- 
tion of a comprehensive report of his own research activities a 
straightforward and even enjoyable undertaking. 



M LA/ERIC solicits reports of locally-conducted research studies and other 
unpublished documents of potential usefulness in foreign-lar^age instruction. 
Accepted reports are catalogued in RIE and made available to the profession 
through the ERIC Documej^ ^production Service (EDRS). 
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